What is SIE?

SIE (Superlinked Inference Engine) is an open-source inference server for AI models. It runs encoders, rerankers, entity extractors, and generation models on your own infrastructure, from a laptop to a production Kubernetes cluster, without managing per-model deployments or paying per-token API costs.

SIE exposes four primitives:

Encode converts text or images to vectors for semantic search and RAG
Score reranks query-document pairs for higher-precision retrieval
Extract pulls entities and structured data from unstructured text
Generate runs text generation on open LLMs you host yourself

100+ models are supported out of the box. The server handles batching, GPU sharing, and model switching automatically. Browse the full model catalog.

SIE is built by Superlinked, the team behind the Superlinked vector compute framework. Read the launch post.

Get Started

I want to…	Go to
Get my first vectors in 2 minutes	Quickstart
Embed text or images	Encode Overview
Rerank search results	Score Overview
Extract entities from text	Extract Overview
Generate text with a small LLM	Generate Overview
Choose the right model	Model Selection Guide
See all 100+ models	Model Catalog
Deploy to production	Deployment Overview
Connect to LangChain or LlamaIndex	Integrations

Why Does SIE Exist?

LLM inference tools are designed for one large model spread across many GPUs. Small model inference is the opposite problem: you run many models (encoders, rerankers, extractors) on one GPU and need fast switching between them.

What makes SIE different from other inference servers:

Compute engine abstraction. SIE wraps PyTorch, SGLang, Flash Attention, and Apple MLX behind four uniform primitives. The server picks the best engine per model automatically.
Multi-model GPU sharing. Many models can share one GPU via LRU eviction. One SIE instance serves any model at query time without pre-loading everything.
Same code, laptop to cloud. The same Docker image runs locally and in a production Kubernetes cluster. There is no separate production mode.
Validated correctness. Every supported model has quality and latency targets checked in CI.

How Does SIE Compare to Alternatives?

	SIE	TEI (HuggingFace)	OpenAI API
Self-hosted	Yes	Yes	No
Multi-model on one GPU	Yes	No (one model per server)	N/A
Encode + Score + Extract + Generate	Yes	Encode only	Encode + generate only
100+ supported models	Yes	Varies	Limited
Open source	Yes	Yes	No
No per-token cost	Yes	Yes	No

See the retrieval strategy benchmark for a worked evaluation of eight retrieval strategies on SIE with measured NDCG@10 numbers.

Frequently Asked Questions

What is SIE used for? SIE is used to generate embeddings for semantic search and RAG pipelines, rerank search results to improve precision, extract entities from unstructured text, and run text generation on small self-hosted LLMs. All of this runs on your own infrastructure. See superlinked.com for more on what you can build.

Does SIE support GPU inference? Yes. SIE runs on CPU or GPU. For production inference at scale, a GPU is strongly recommended. See Hardware and Capacity for GPU sizing guidance.

How many models can SIE run at the same time? SIE loads models on demand and evicts the least-recently-used models when GPU memory fills up. An L4 GPU (24GB) holds several standard models resident at once; the exact count depends on model size and batch settings. The full catalog stays addressable at query time: models load on demand, and each requested model still has to fit in the GPU’s memory to load.

Is SIE open source? Yes. SIE is open source and available on GitHub. The core inference server is free to use. Superlinked also offers managed cloud deployment. Contact us to learn more.

How is SIE different from the Superlinked framework? The Superlinked framework is a higher-level Python SDK for building multi-attribute search and recommendation systems. SIE is the inference layer underneath it. You can use SIE standalone or as part of a full Superlinked stack.